Multiplexed, multimodal conferencing

ABSTRACT

A method for managing multiple electronic conference sessions. The method includes a computer processor identifying a plurality of conference sessions that a user is attending, wherein the plurality of conference sessions includes a first session and a second session. The method further includes a computer processor identifying one or more triggers that indicate an occurrence of an event in at least one of the plurality of conference sessions. The method further includes a computer processor determining that the user is an active participant in at least the first session of the plurality of conference sessions that the user is attending. The method further includes a computer processor detecting at least one trigger of the one or more identified triggers, within the second session of the plurality of conference sessions that the user is attending.

BACKGROUND OF THE INVENTION

The present invention relates generally to the field oftelecommunication, and more particularly to teleconferencing.

A telephone or audio conference (i.e., teleconference) enables multipleparticipants to communicate with the other participants of theteleconference, or optionally, mute a connection and attend theteleconference in “listen only” mode. Video conferences (e.g., videotelephony) add visual elements to a conference session. Sometechnologies utilize computer systems to communicate the visual elements(e.g., charts, graphs, videos, pictures, etc.) and other digitalelements (e.g., digital audio files, animated graphics, etc.), whilestandard telephony communicates the verbal (e.g., audio) portion of theconference session.

Businesses can utilize teleconferences to communicate directly withmultiple individual (e.g., of the business, of a supplier, of a client,etc.) dispersed across various geographical location without incurringthe time, the expense, and the disruption associated with travel forin-person interactions. Mobile collaboration systems combine the use ofvideo, audio, and on-screen drawing capabilities using various hand-heldelectronic devices to broadcast over secure networks, enablingmulti-party conferencing in real-time, independent of location. With theadvent of Cloud computing systems and services, digital teleconferencingprovides a more secure mobile collaboration system where user mayexchange data, files, images, videos, and in some instances,collaboration software can allow participants in different locations tointeract with the data, files, images, and videos (e.g., projectmanagement, online proofing, etc.).

Teleconferencing is not limited to a business environment. Education andmedicine also benefit from teleconferencing. For example, educationalinstitutions offer on-line classes that allow students to obtaindegrees. In some cases, a student utilizes the Internet to participatein a real-time classroom lecture that may include an instant messaging(IM) function or audio function to ask questions. In other cases,classes and lectures may be prerecorded and a student accesses theinformation in an on-demand mode over the Internet.

SUMMARY

According to aspects of the present invention, there is a method,computer program product, and/or system for managing multiple electronicconference sessions. The method includes one or more computer processorsidentifying a plurality of conference sessions that a user is attending,wherein the plurality of conference sessions includes a first sessionand a second session. The method further includes one or more computerprocessors identifying one or more triggers that indicate an occurrenceof an event in at least one of the plurality of conference sessions. Themethod further includes one or more computer processors determining thatthe user is an active participant in at least the first session of theplurality of conference sessions that the user is attending. The methodfurther includes one or more computer processors detecting at least onetrigger of the one or more identified triggers, within the secondsession of the plurality of conference sessions that the user isattending.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a distributed computing environment, in accordancewith an embodiment of the present invention.

FIG. 2 depicts a flowchart of the operational steps of a conferencesetup program, in accordance with an embodiment of the presentinvention.

FIG. 3 depicts a flowchart of the operational steps of a sessionmonitoring program, in accordance with an embodiment of the presentinvention.

FIG. 4 is a block diagram of components of a computer, in accordancewith an embodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention recognize that a user may need thecapability to attend multiple audio conferences, video conferences, andcollaboration events, herein identified as sessions, at the same time.However, embodiments of the present invention also recognize that a usermay need to isolate one session from another session. For example, auser may isolate sessions to maintain focus, minimize cross-talk betweensession that can be disruptive to other participants, and maintainsecurity (e.g., avoid unauthorized sharing of information). Someembodiments of the present invention may utilize a mobile device such asa smartphone. Other embodiments of the present invention utilize voiceover Internet protocol (VoIP) to communicate via a computing device(e.g., desktop computer). In another embodiment of the presentinvention, the capabilities of a computing device and a digitaltelephony system may be integrated to enable a user to attend andmonitor multiple sessions.

Embodiments of the present invention may be utilized in a plurality ofenvironments and functions. For example, attending and monitoringmultiple sessions may be utilized by: businesses, students, educators,researchers, government agencies, medical professionals, hospitals andthe like. Embodiments of the present invention obtain information thatis relevant to a user, an organization, a project, etc. and utilizes theobtained information as the basis for triggers that can be detectedwithin one or more sessions. Embodiments of the present invention mayutilize a variety of semantic conversion programs to provide streamingtext transcripts of the proceeding of each session. In addition, anembodiment of the present invention may determine information related toeach user attending a session (e.g., a name, an organization, a statusor mode, such as participating, muted, off-line, auditing, monitoring,and observing) and include the related information within the streamingtranscript. In some embodiments of the present invention, theinformation provided within streaming text transcript may vary by userand the device of the user. In an example, if a user is utilizing anelectronic device that constrains the user to audio and text, thestreaming text transcript may include descriptions of visual elements ofthe presentation such as gestures of a presenter.

In addition, embodiments of the present invention monitor the real-timetranscripts of each session, scanning (e.g., detecting) for the presenceof one or more triggers. Embodiments of the present invention provide auser an indication as to which session includes the detected triggersand provides the user a method to automatically modify the connectionand/or user mode of a session when another session is selected. Inanother embodiment, the present invention may provide a user an optionto save a streaming transcript of a session. Some embodiments of thepresent invention enable a user may have two or more active sessions.Other embodiments of the present invention enable a user may link two ormore sessions.

In addition, embodiments of the present invention may utilize othersoftware to determine additional information corresponding to thecontent (e.g., verbal, text, graphical, physical) presented during asession that is included in the streaming transcript. For example,embodiments of the present invention may utilize: cognitive analysis,contextual analysis, and analytics to determine the additionalinformation corresponding to the content within a monitored session.Similarly, embodiments of the present invention may utilize: cognitiveanalysis, contextual analysis, and analytics to further analyze:gestures, comments, graphics, sounds, video, etc., to determine relatedinformation (e.g., citations, regulations, definitions, etc.). Therelated information may be accessed via an universal resource locator(URL) that is embedded (e.g., hyperlinked) in the streaming transcript.Some embodiments of the present invention may utilize cognitiveanalysis, contextual analysis, analytics, and other databases todetermine the additional trigger that are associated with a user.Example embodiments of the present invention may accesses a companydatabase to determine the names of peers and reporting structure of auser and include the determined names as additional triggers to detectwithin a streaming transcript.

Further embodiments of the present invention may utilize semanticconversion, cognitive analysis, contextual analysis, and analyticssoftware that analyze information from prerecorded sessions and/orinformation feeds (e.g., on-demand lectures, new stories, digitaltranscripts), to provide a user access to information (e.g., to sort, tolocate, to display, etc.) within the linked content of a streaming texttranscript. Some embodiments of the present invention, the may utilizeanalytical software that can process information at an increasedthroughput (e.g., fast-forward). In other embodiments of the presentinvention, a recording of a conference session (e.g., presentation,proceedings, etc.) and a corresponding streaming text transcript aresaved to a conferencing database for future access.

The descriptions of the various scenarios, instances, and examplesrelated to the present invention have been presented for purposes ofillustration but are not intended to be exhaustive or limited to theembodiments disclosed.

The present invention will now be described in detail with reference tothe Figures. FIG. 1 is a functional block diagram illustratingdistributed computing environment 100 in an embodiment, in accordancewith the present invention. An embodiment of distributed computingenvironment 100 may include: user device 102, client device 115, clientdevice 120, and client device 125, all interconnected over network 110.Many modifications to the depicted environment may be made by thoseskilled in the art without departing from the scope of the invention asrecited by the claims.

User device 102, client device 115, client device 120, and client device125 may be laptop computers, tablet computers, netbook computers,personal computers (PC), desktop computers, personal digital assistants(PDA), smart phones, wearable devices (e.g., digital eyeglasses, smartglasses, a smart watch), or any programmable computer systems known inthe art. In certain embodiments, user device 102, client device 115,client device 120, and client device 125 represent computer systemsutilizing clustered computers and components (e.g., database servercomputers, application server computers, etc.) that act as a single poolof seamless resources when accessed through network 110, as is common indata centers and with cloud-computing applications. In general, userdevice 102, client device 115, client device 120, and client device 125are representative of any programmable electronic device or combinationof programmable electronic devices capable of executing machine readableprogram instructions and communicating with users of user device 102,client device 115, client device 120, and client device 125, via network110. User device 102, client device 115, client device 120, and clientdevice 125 may include components, as depicted and described in furtherdetail with respect to FIG. 4, in accordance with embodiments of thepresent invention.

User device 102 includes user information 104, semantic conversionprogram 106, user interface (UI) 122, conference setup program 200,session monitoring program 300, and various programs and databases(e.g., a website interface, presentation software, telephony software,voice over Internet protocol (VoIP) software, conferencing software, acalendar program, audio/video playback software, etc.) (not shown). Insome embodiments, user device 102 may include additional programs thatprovide cognitive, analytic, and contextual analysis capabilities tosemantic conversion program 106. In other embodiments, user device 102may utilize additional programs that provide cognitive, analytic, andcontextual analysis capabilities to semantic conversion program 106 on acomputing system accessed via network 110.

In one embodiment, user information 104 contains user information, userpreferences, predefined triggers, predefined responses, conferencesystem setups, controls (e.g., phone numbers, access codes, hot-keycombinations, etc.), and other information utilized by conference setupprogram 200. For example, a user preference may predefine how userdevice 102 responds to a haptic event (e.g., vibrating) to identifywhich session includes the occurrence of a trigger. In a scenario, userinformation 104 includes user preferences that may dictate whichcapabilities of semantic conversion program 106 are utilized for eachsession. In an example, user information 104 may dictate that semanticconversion program 106 annotates a text transcript with a namecorresponding to a participant of a portion of the transcript.

In another embodiment, user information 104 includes information (e.g.,triggers) that session monitoring program 300 utilizes to detectoccurrences of information within the output of semantic conversionprogram 106. In one scenario, user information 104 includes informationand/or triggers that are generalized and may be identified within one ormore sessions. In an additional example, user information 104 mayinclude organizational information (e.g., names, departments, reportingstructures, calendar entries, etc.) associated with a user of userdevice 102. In another scenario, user information 104 may includeinformation and/or triggers that are defined for a session (e.g.,primary triggers). In an example, user information 104 includes thetriggers: proposal #309432A2, CAD drawings, and government regulations,which are defined for a session that utilizes access code 153401*. Inanother scenario, user information 104 may include one or moreconfidence intervals utilized to determine the detection of a trigger orother analysis of a session (e.g., fuzzy triggers). In some embodiments,user information 104 includes additional details related to the user ofuser device 102 that may be utilized to generate contextual triggers(e.g., secondary). In an example, the additional details of a userinformation may include, but not limited to: areas of expertise, skills,social-media information, educational background, certifications, andprofessional networking contacts.

In one embodiment, semantic conversion program 106 may be aspeech-to-text conversion program that provides a streaming transcript(e.g., text) of a session to a user of user device 102. In someembodiments, semantic conversion program 106 includes identifiers (e.g.,annotation, participation mode, etc.) for one or more users attending aconference session within a streaming transcript. In another embodiment,semantic conversion program 106 includes image and gesture recognition.In a further embodiment, semantic conversion program 106 may becomprised of a suite of programs that includes: semantic, cognitive,analytical, contextual analysis, image processing, gesture recognition,database search capabilities and the like. In some scenarios, semanticconversion program 106 may utilize fuzzy logic and confidence intervalsidentified within user information 104 when analyzing a session. In oneexample, semantic conversion program 106 follows a gesture by apresenter, within a conference session, to the indicated portion of thescreen and semantic conversion program 106 further analyzes theindicated portion of the screen and includes a description of theanalysis in the text transcript. In another example, semantic conversionprogram 106 accesses various Intranet and Internet databases to obtaininformation and/or information links that can be associated elements ofa session that are indicated in a streaming transcript. In someembodiments, semantic conversion program 106 utilizes a pluralityprograms and devices, accessible via network 110, to enable conferencesetup program 200 and/or session monitoring program 300.

In another embodiment, semantic conversion program 106 may includemulti-lingual conversion capabilities. Semantic conversion program 106may augment a transcript of a session with additional information (e.g.,citations, regulations, definitions, news articles, etc.) and/orcontrols to access the additional information. The additionalinformation may include various indicators, such as highlights, coloredhyperlinks, speaker names, file links, graphical control elements, etc.,within a real-time transcript of a session as opposed to describingnon-verbal portions (e.g., charts, graphs, pictures, etc.) of a session.For example, highlighted information within a session transcript maygenerate a temporary overlay that includes additional details when auser of user device 102 hovers the mouse pointer (e.g., mouse-over) overthe highlighted text. In some embodiments, semantic conversion program106 interfaces (e.g., synchronizes) with a recording function of theconferencing software. In an example, scrolling through a texttranscript created by semantic conversion program 106 may move (e.g.,rewind, fast-forward) a recorded portion of a session to match the textselected by a user.

In an alternative embodiment, semantic conversion program 106 exists onnetwork 110 as a service by a provider of the conferencing system. In ascenario, semantic conversion program 106 streams information to eachuser attending a teleconference or video conference. In some scenarios,semantic conversion program 106 may interface with one or more supportprograms (e.g., an image processing program, a gesture recognitionprogram, cognitive analysis program, a contextual analysis program, ananalytical program, a database search program, etc.) on user device 102.In other scenarios, semantic conversion program 106 interfaces with oneor more support program accessible via network 110. In other scenarios,the transcripts (e.g., streaming text) of a session are limited to theusers of the conferencing system that subscribe to semantic conversionprogram 106 of the conference system provider.

In one embodiment, user device 102, client device 115, client device120, and client device 125 communicate through network 110. Network 110can be, for example, a local area network (LAN), a telecommunicationsnetwork, a wireless local area network (WLAN), such as an intranet, awide area network (WAN), such as the Internet, or any combination of theprevious and can include wired, wireless, or fiber optic connections. Ingeneral, network 110 can be any combination of connections and protocolsthat will support communications between user device 102, client device115, client device 120, and client device 125, in accordance withembodiments of the present invention. In another embodiment, network 110operates locally via wired, wireless, or optical connections and can beany combination of connections and protocols (e.g., near fieldcommunication (NFC), laser, infrared, etc.).

UI 122 accepts input from a plurality of input/output (I/O) devices (notshown) including, but not limited to, a tactile sensor interface (e.g.,a touch screen, a touchpad) or a natural user interface (e.g., voicecontrol unit, motion capture device, eye tracking, cyberglove, head-updisplay etc.). In addition to the audio and visual interactions thatuser device 102 may include, a peripheral, such as a cyberglove that mayprovide haptic feedback (e.g., vibrations) to a user. An I/O deviceinterfacing with a UI 122 may be connected to a user device 102 whichmay operate utilizing wired (e.g., USB port) or wireless networkcommunications (e.g., infrared, NFC, etc.). For example, an I/O devicemay be a peripheral, such as a keyboard, a mouse, a trackball, and aclick wheel that provides input from a user. A user may utilize UI 122to interact with conference setup program 200 and session monitoringprogram 300.

In some embodiments, UI 122 may automatically reconfigure based on thetype of conference sessions that user device 102 accesses. In otherembodiments, UI 122 may automatically reconfigure based on one or moreconstraints. In an example, user device 102 is bandwidth constrained andcannot present audio, video, and text together. UI 122 may utilize auser preference to reconfigure to text and audio only. Subsequently,session monitoring program 300 detects the modifications thepresentation of the conferences by UI 122 and session monitoring program300 adjust one or more flags presented to a user of user device 102 whena trigger is activated. In other embodiments, UI 122 includes thewindows or other graphical control elements that may display apresentation of a session and a respective window for the streaming texttranscript for a session. In some scenarios, UI 122 responds to theoccurrence of a trigger based on a user preference. In an example, UI122 increases the size of the window for a session that includes anoccurrence of a trigger and positions the window in proximity to awindow that includes a session where the user is a participant.

Conference setup program 200 enables a user of user device 102 toconnect to a plurality of conference sessions. Conference setup program200 monitors the presentations (e.g., proceedings) of each conferenceutilizing semantic conversion program 106 to generate streaming texttranscripts of the monitored sessions. Conference setup program 200 mayexecute multiple instances of semantic conversion program 106. In someembodiments, conference setup program 200 utilizes an instance ofsemantic conversion program 106 that includes: cognitive analysis,contextual analysis, and/or analytics capabilities. In addition,conference setup program 200 interfaces with session monitoring program300 to detect information within a monitored session that may activateone or more triggers within user information 104 and provide anindication (e.g., audio, visual, haptic) to the user of user device 102when a trigger is activated within a monitored session. Conference setupprogram 200 may execute at the same time as session monitoring program300.

In other embodiments, one or more sessions may be prerecorded audioand/or visual content as opposed to real-time sessions (e.g., events),which are obtained as inputs from a user. In other embodiments, one ormore sessions may be prerecorded audio and/or visual content as opposedto real-time sessions (e.g., events), which are embedded within contentof a conferencing session. For example, conference setup program 200 mayaccess a training video, news clip, on-demand education lecture, etc. Ina further embodiment, conference setup program 200 may utilize anaudio/video playback program that is capable of variable speed (e.g.,fast-forward) playback of prerecorded content. Conference setup program200 may utilize an instance of semantic conversion program 106 that iscapable of analyzing the audio/video content at an accelerated playbackrate.

In an embodiment, session monitoring program 300 utilizes informationprovided by conference setup program 200 and semantic conversion program106 to monitor multiple sessions and provide the user of user device 102an indication when a trigger is detected in a session. In addition,session monitoring program 300 may modify the conference sessionconnections when a user of user device 102 indicates that the userchanges the conference session in which the user participates and whichthe user monitors. In a further embodiment, session monitoring program300 may enable a user of user device 102 to interface with more than onesessions. In some embodiments, session monitoring program 300 provides autility for a user to link (e.g., three-way call) two sessions that theuser of user device 102 attends (e.g., monitors, participates). In otherembodiments, session monitoring program 300 provides a utility to accesscontent (e.g., session content, pre-recorded content, etc.) withoutmodifying one or more conference session connections.

FIG. 2 is a flowchart depicting operational steps for conference setupprogram 200, a program for connecting a user to a plurality ofconference sessions and interfacing with session monitoring program 300,in accordance with embodiments of the present invention. In addition,conference setup program 200 may utilize one or more instances ofsemantic conversion program 106 to display streaming text correspondingto the presentation of respective sessions.

In step 202, conference setup program 200 obtains conference attendanceinformation and associated user information. In one embodiment,conference setup program 200 obtains conference attendance informationfor a session from user information 104. In an example, conferenceattendance information obtained by conference setup program 200 mayinclude: session login information, encryption protocol, conferencesession command codes (e.g., mute, transfer, raise hand, etc.),attendees, keywords, user defined triggers, contextual based triggers,and other contextual information. In another embodiment, conferencesetup program 200 obtains user information from user information 104,which may be provided to a session. In an example, conference setupprogram 200 may provide a first conference session a name of a user, acompany of a user, a title of a user, an e-mail for a user, and alocation of a user. Whereas, conference setup program 200 may restrictthe user information provided to a second session to the name of a user.

In some embodiments, conference setup program 200 obtains additionalinformation for a user from user information 104 such as: areas ofexpertise, skills, social-media information, educational background,certifications, and professional networking contacts. In one scenario,conference setup program 200 may include additional information of auser as participant information for a session, such as informationaccessible by other participants of the session. In another scenario,conference setup program 200 may provide additional associated with auser to semantic conversion program 106. In some scenarios, conferencesetup program 200 may provide additional information for a user to botha session and semantic conversion program 106.

Referring to step 202 in other embodiments, conference setup program 200determines the capabilities of user device 102. In one scenario,conference setup program 200 determines whether user device 102 includesan instance of semantic conversion program 106 and the capabilities ofthe instance of semantic conversion program 106. In an example,conference setup program 200 determines whether semantic conversionprogram 106 includes: cognitive analysis, contextual analysis, and/oranalytics capabilities. In another scenario, conference setup program200 determines the user interface capabilities associated with userdevice 102. In an example, conference setup program 200 may determine: ascreen size (e.g., dual monitors, smart phone screen, etc.) of userdevice 102, an user interface for user device 102 (e.g., keyboard andmouse, touch screen, voice activated, etc.), and a connection speed foruser device 102 (e.g., 10 GB/second Ethernet, 2 MB/second wireless,etc.).

In step 204, conference setup program 200 connects to a conferencingsystem and determines whether a user participates in a session. In oneembodiment, conference setup program 200 monitors user device 102 todetermine when a user attends (e.g., logs into) a session. In anotherembodiment, conference setup program 200 accesses a calendar of a userand identifies one or more sessions within a window of time. Conferencesetup program 200 obtains information (e.g., login information) relatedto accessing the one or more identified sessions from the calendar of auser. Subsequently, conference setup program 200 establishes aconnection to the one or identified sessions within a calendar of auser. In addition, conference setup program 200 determines which one ormore sessions that a user is an active participant as opposed to apassive observer (e.g., through monitoring of the user, by analyzingconference session connection parameters). In one embodiment, conferencesetup program 200 determines whether a user participates in a sessionbased on the conference session command codes input by a user for asession. In another embodiment, conference setup program 200 determinewhether a user participates in a session based on information associatedwith a graphic control element (e.g., GUI, window, etc.) of a session.In an example, conference setup program 200 may determine that a userparticipates in a session that executes (e.g., displays) within amaximized window and that the user observers (e.g., monitors) one ormore session that execute within a minimized window. In someembodiments, conference setup program 200 may activate a record functionwithin the conferencing software utilized by conference setup program200.

In step 206, conference setup program 200 activates a semanticconversion program and displays text associated with a session. In anembodiment, conference setup program 200 indicates that semanticconversion program 106 generates a real-time, streaming text transcriptof a presentation for a session. In some embodiments, conference setupprogram 200 activates an instance of semantic conversion program 106 foreach session attended by a user. In one embodiment, conference setupprogram 200 activates an instance of semantic conversion program 106that monitors a session and semantic conversion program 106 generates astreaming text transcript of the presentation within the session. Inanother embodiment, conference setup program 200 utilizes userpreferences within user information 104 to determine which capabilities(e.g., cognitive, analytical, contextual, language translation,annotation, highlighting, hot-spots, etc.) of semantic conversionprogram 106 are utilized to generate a text transcript of a session.

In another embodiment, conference setup program 200 generates a windowfor each text transcript generated by semantic conversion program 106.In one scenario, conference setup program 200 generates another windowfor each session that includes a session window and the respectivewindow displaying a text transcript (e.g., places the two windows inproximity) of the session generated by semantic conversion program 106.In another embodiment, conference setup program 200 links a sessionwindow to a window that displays the respective text transcriptgenerated by semantic conversion program 106 for the session (e.g.,places the two windows in proximity). In some embodiments, conferencesetup program 200 hides a window associated with a session that the useris a participant. In other embodiments, conference setup program 200enables a user to size and position a window containing a texttranscript that is associated with a session independently of the sizeand status of session window.

In decision step 208, conference setup program 200 determines whether anattended conference session ends. In one embodiment, conference setupprogram 200 determines that an attended session ends when a windowassociated with a session is closed. In another embodiment, conferencesetup program 200 determines that a session ends when session setupprogram detects that a user initiates a command code (e.g., close,logout, etc.) that terminates a session connection. In some embodiments,conference setup program 200 determines that a session ends when anexternal command or message (e.g., host disconnected from network, endof session, etc.) is issued. In an example, a user of client device 120hosts a presentation (e.g., hosts a session). Conference setup program200 determines that the presentation is over when client device 120disconnects from network 110. In another example, conference setupprogram 200 detects an “end of presentation” message in a texttranscript for a session generated by semantic conversion program 106.

In decision step 208, responsive to determining that an attended sessiondid not end (No branch, decision step 208), conference setup program 200communicates with session monitoring program 300 (step 210).

In step 210, conference setup program 200 communicates with sessionmonitoring program 300. In one embodiment, conference setup program 200communicates information (e.g., user preferences, triggers, conferencesession command codes, etc.) from user information 104 that is relatedto one or more sessions that is attended by a user to session monitoringprogram 300. In another embodiment, conference setup program 200communicates text transcripts for each session to session monitoringprogram 300. In some embodiments, conference setup program 200communicates additional contextual analysis of information within userinformation 104 to session monitoring program 300. In other embodiments,conference setup program 200 communicates addition content andinformation generated by additional programs (e.g., cognitive analysis,analytics, contextual analysis, image recognition, etc.) and/or semanticconversion program 106 based on further analysis of the text transcriptsfor each session to session monitoring program 300.

Referring to decision step 208, responsive to determining that amonitored session ends (Yes branch, decision step 208), conference setupprogram 200 communicates with a user (step 212).

In step 212, conference setup program 200 communicates with a user.Conference setup program 200 communicates with a user via UI 122. In oneembodiment, conference setup program 200 may save a text transcript of asession. In one scenario, conference setup program 200 determines that asystem default dictates that a text transcript for a session is saved.In another scenario, conference setup program 200 determines that a userpreference within user information 104 overrides the system default.Subsequently, conference setup program 200 provides a user of userdevice 102 an option to save the text transcript for a session that auser ceases to attend. In another embodiment, conference setup program200 may advise a user that a session abnormally ended. In an example,user device 102 cannot communicate with client device 115 via network110 during a session hosted by client device 115. A user may requestthat conference setup program re-establish a connection (step 204) tothe session hosted by client device 115. In some embodiments, conferencesetup program 200 saves a text transcript of a session to user device102. In other embodiments, conference setup program 200 saves a texttranscript of a session to a database accessible via network 110. In afurther embodiment, conference setup program 200 saves a recordedsession and a corresponding text transcript generated by semanticconversion program 106.

In decision step 214, conference setup program 200 determines whether tomonitor the remaining sessions attended by a user. In one embodiment,conference setup program 200 obtains an indication from a user, via UI122, to monitor session that are active and that the user attends. Inanother embodiment, conference setup program 200 determines that a userpreference within user information 104 indicates that a user monitorsattended sessions.

In decision step 214, responsive to determining to monitor the one ormore remaining session (Yes branch, decision step 214), conference setupprogram 200 communicates with session monitoring program 300 (step 210).

Referring to decision step 214, responsive to determining not to monitorthe remaining attended sessions (No branch, decision step 214),conference setup program 200 terminates. If conference setup program 200determines that no attended sessions exist (No branch, decision step214), then conference setup program 200 terminates.

FIG. 3 is a flowchart depicting operational steps for session monitoringprogram 300, a program for monitoring multiple sessions and signaling auser when one or more triggers occur within one of the one or moresessions that a user attends. Session monitoring program 300 may utilizeinputs from multiple instances of semantic conversion program 106 and/orconference setup program 200.

In step 301, session monitoring program 300 analyzes a session anddetects an occurrence of a trigger. In one embodiment, sessionmonitoring program 300 analyzes a text transcript generated by semanticconversion program 106 corresponding to a session and detects theoccurrence of one or more triggers identified within user information104. In one scenario, session monitoring program 300 may detect atrigger based on general user information (e.g., user name, an employerof the user, etc.) or other generalized triggers defined by a user. Inanother scenario, session monitoring program 300 detects the occurrenceof one or more triggers defined by a user for a session.

In some embodiments, session monitoring program 300 further analyzes atext transcript, generated by semantic conversion program 106, for asession. In one example, session monitoring program 300 may utilize adictionary function to identify text that is similar (e.g., synonyms,homonyms, common misspellings, etc.) to the text of a trigger. Inanother example, session monitoring program 300 analyzes elementsidentified (e.g., highlighted, hyperlinked, etc.) within a sessiontranscript to detect the occurrence of a trigger. In a furtherembodiment, session monitoring program 300 may utilize: cognitiveanalysis, contextual analysis, analytics, image processing, voicerecognition software, etc., to determine information that relates to atrigger. In an example, session monitoring program 300 may determine thenames of coworkers of a user, and detect the names of the coworkers astriggers when a trigger is the name of the user. In another example,session monitoring program 300 determines triggers (e.g., project name,individuals assigned to proposal, etc.) for a session that is associatedwith a business proposal based on a user trigger of “client name” for asession. In addition, session monitoring program 300 may utilize fuzzylogic and confidence intervals identified within user information 104when analyzing a session for an occurrence of a trigger.

Referring to step 301, in an alternate embodiment, session monitoringprogram 300 utilizes instances of various cognitive analysis, contextualanalysis, analytics, image processing, and voice recognition software tofurther analyze: gestures, comments, graphics, sounds, video, etc. of asession to determine related information (e.g., citations, regulations,definitions, news articles, etc.) within the presentation (e.g.,proceedings), as opposed to a text transcript provided by semanticconversion program 106. Subsequently, session monitoring program 300 mayutilize the related information to detect the occurrence of a trigger.In addition, session monitoring program 300 may utilize fuzzy logic andconfidence intervals identified within user information 104 whenanalyzing a session for related information. In some embodiments,session monitoring program 300 utilizes the analysis of the texttranscript generated by semantic conversion program 106 and analysis ofa presentation of a session to detect the occurrence of a trigger.

In decision step 302, session monitoring program 300 determines whethera trigger occurs within a session. In response to determining that atrigger occurs within a session (Yes branch, decision step 302), sessionmonitoring program 300 identifies the session that is associated with anoccurrence of a trigger (step 304).

In step 304, session monitoring program 300 identifies a conferencesession that is associated with an occurrence trigger. In oneembodiment, session monitoring program 300 applies a visual indicationto a window of a session that includes an occurrence of a trigger. Inone scenario, session monitoring program 300 highlights a window withinUI 122 that includes a session where one or more triggers occurred. Inanother scenario, session monitoring program may change the size of thewindow that includes a session where one or more triggers occurred. In afurther embodiment, session monitoring program 300 adjusts a visualindication of a window that includes a session that includes anoccurrence of a trigger based on the number of unique triggers thatoccur, the frequency of the triggers that occur, and/or the analysis(e.g., cognitive, contextual, analytics, confidence level) associatedwith the one or more triggers that occur within a session. In anexample, session monitoring program 300 may respond to a single instanceof a trigger by applying a constant highlight to a window for a sessionas opposed to modifying the strobe rate of the highlight of window baseon the number of triggers that occur within a session.

In an alternate embodiment, session monitoring program 300 determinesthat user device 102 (e.g., a smartphone) has a constrained displayarea. Session monitoring program 300 may determine that a userparticipates in a session utilizes the full screen area of user device102. In one scenario, session monitoring program 300 includes anindication within UI 122 as to which session includes an occurrence of atrigger. In another scenario, session monitoring program 300 utilizesanother component of user device 102 to advise a user that a triggeroccurred within another session. For example, session monitoring program300 may initiate a haptic event (e.g., the smartphone vibrates) tosignal the user that a trigger occurred within another session.

Referring to decision step 302, responsive to determining that a triggerdid not occur within a session (No branch, decision step 302), sessionmonitoring program 300 determines whether a user changes to anothersession (decision step 306).

In decision step 306, session monitoring program 300 determines whethera user changes sessions. In one embodiment, session monitoring program300 determines that a user changes sessions when the focus of the userchanges. In one scenario, session monitoring program 300 determines thata user minimizes a window that includes a session that the user is aparticipant and enlarges a window associated with another session. Inanother scenario, session monitoring program determines that a userchanges sessions when the user utilizes UI 122 to manually interfacewith a session (e.g., activate listen-only mode, raise a hand, changethe status of a user, etc.). In another embodiment, session monitoringprogram 300 determines that a user changes sessions based on a responseto an event. In one scenario, session monitoring program 300 determinesthat one or more triggers occur within a text transcript for a session.In another scenario, session monitoring program 300 determines that oneor more triggers occurred within the analysis of a presentation within asession. In some embodiments, session monitoring program 300 maydetermine that a user attends more than one session. In otherembodiments, session monitoring program 300 determines that a userchanges session when the user selects the window associated with asession.

In decision step 306, responsive to determining that a user does notchange sessions (No branch, decision step 306), session monitoringprogram 300 loops while continuing to detect the occurrence of a triggerwithin other sessions (step 301).

Referring to decision step 306, responsive to determining that a userchanges to another session (Yes branch, decision step 306), sessionmonitoring program 300 modifies session connections (step 308).

In step 308, session monitoring program 300 modifies conference sessionconnections. In one embodiment, session monitoring program 300determines that a user changes focus (e.g., participate, observe,monitor) between sessions. In one scenario, session monitoring program300 utilizes user information 104 to obtain command codes for therespective sessions and session monitoring program 300 executes thecommand codes for a session via a conferencing system (e.g.,conferencing software, telephony system, etc.). In another scenario,session monitoring program 300 utilizes UI 122 to obtain input (e.g.,select conferencing option) from a user prior to selecting and executinga command code for a session via a conferencing system. In anotherembodiment, session monitoring program 300 determines that a usermodifies the access, status, and/or mode of one or more sessions. In oneexample, session monitoring program 300 determines that a user ends(e.g., leaves, closes, logs out, etc.) a session. In another example,session monitoring program 300 determines that a user initiates athree-way link between user device 102 and a session hosted by clientdevice 120 and another session hosted by client device 125.

In step 310, session monitoring program 300 communicates with conferencesetup program 200. In one embodiment, session monitoring program 300communicates the status of two or more sessions when session monitoringprogram 300 determines that the focus of the user changes. In anotherembodiment, session monitoring program 300 communicates that a user endsa session. In some embodiments, session monitoring program 300communicates information related to: detected triggers (e.g., frequencyof occurrence, primary triggers, secondary triggers, fuzzy triggers),the timing of trigger occurrences within a session, and sessionstatistics (e.g., duration, attendees).

FIG. 4 depicts computer system 400, which is representative of userdevice 102, client device 115, client device 120, and client device 125.Computer system 400 is an example of a system that includes software anddata 412. Computer system 400 includes processor(s) 401, memory 402,cache 403, persistent storage 405, communications unit 407, input/output(I/O) interface(s) 406, and communications fabric 404. Communicationsfabric 404 provides communications between memory 402, cache 403,persistent storage 405, communications unit 407, and input/output (I/O)interface(s) 406. Communications fabric 404 can be implemented with anyarchitecture designed for passing data and/or control informationbetween processors (such as microprocessors, communications and networkprocessors, etc.), system memory, peripheral devices, and any otherhardware components within a system. For example, communications fabric404 can be implemented with one or more buses or a crossbar switch.

Memory 402 and persistent storage 405 are computer readable storagemedia. In this embodiment, memory 402 includes random access memory(RAM). In general, memory 402 can include any suitable volatile ornon-volatile computer readable storage media. Cache 403 is a fast memorythat enhances the performance of processor(s) 401 by holding recentlyaccessed data, and data near recently accessed data, from memory 402.

Program instructions and data used to practice embodiments of thepresent invention may be stored in persistent storage 405 and in memory402 for execution by one or more of the respective processor(s) 401 viacache 403. In an embodiment, persistent storage 405 includes a magnetichard disk drive. Alternatively, or in addition to a magnetic hard diskdrive, persistent storage 405 can include a solid-state hard drive, asemiconductor storage device, a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM), a flash memory, or any othercomputer readable storage media that is capable of storing programinstructions or digital information.

The media used by persistent storage 405 may also be removable. Forexample, a removable hard drive may be used for persistent storage 405.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer readable storage medium that is also part of persistent storage405. Software and data 412 are stored in persistent storage 405 foraccess and/or execution by one or more of the respective processor(s)401 via cache 403 and one or more memories of memory 402. With respectto user device 102, software and data 412 includes user information 104,semantic conversion program 106, user interface (UI) 122, conferencesetup program 200, session monitoring program 300, and various programs(not shown). With respect to client device 115, client device 120, andclient device 125 software and data 412 includes various (programs notshow).

Communications unit 407, in these examples, provides for communicationswith other data processing systems or devices, including resources ofuser device 102, client device 115, client device 120, and client device125. In these examples, communications unit 407 includes one or morenetwork interface cards. Communications unit 407 may providecommunications through the use of either or both physical and wirelesscommunications links. Program instructions and data used to practiceembodiments of the present invention may be downloaded to persistentstorage 405 through communications unit 407.

I/O interface(s) 406 allows for input and output of data with otherdevices that may be connected to each computer system. For example, I/Ointerface(s) 406 may provide a connection to external device(s) 408,such as a keyboard, a keypad, a touch screen, and/or some other suitableinput device. External device(s) 408 can also include portable computerreadable storage media, such as, for example, thumb drives, portableoptical or magnetic disks, and memory cards. Software and data 412 usedto practice embodiments of the present invention can be stored on suchportable computer readable storage media and can be loaded ontopersistent storage 405 via I/O interface(s) 406. I/O interface(s) 406also connect to display 409.

Display 409 provides a mechanism to display data to a user and may be,for example, a computer monitor. Display 409 can also function as atouch screen, such as the display of a tablet computer or a smartphone.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions. The descriptions of the various embodimentsof the present invention have been presented for purposes ofillustration, but are not intended to be exhaustive or limited to theembodiments disclosed. Many modifications and variations will beapparent to those of ordinary skill in the art without departing fromthe scope and spirit of the invention. The terminology used herein waschosen to best explain the principles of the embodiment, the practicalapplication or technical improvement over technologies found in themarketplace, or to enable others of ordinary skill in the art tounderstand the embodiments disclosed herein.

What is claimed is:
 1. A method for managing multiple electronicconference sessions, the method comprising: identifying, by one or morecomputer processors, a plurality of conference sessions that a user isattending utilizing a computing device, wherein the plurality ofconference sessions includes a first session and a second session;identifying, by one or more computer processors, one or more predefinedtriggers of the user, respectively associated with the conferencessessions that the user is attending; generating, by one or more computerprocessors, through a real-time analysis of a presentation correspondingto a conference session that the user is attending, a streaming texttranscript based on content spoken during the conference session and atleast one element of interpreted content that occurs during theconference session; determining, by one or more computer processors,that an interface corresponding to the conference session includes afirst graphical control element associated with presenting theconference session and a second graphical control element associatedwith a streaming text transcript corresponding to the conferencesession; and detecting, by one or more computer processors, anoccurrence of at least one trigger of the one or more predefinedidentified triggers, within the generated streaming text transcriptcorresponding to the second session of the plurality of conferencesessions that the user is attending.
 2. The method of claim 1, wherein:(i) at a least a portion of the first session and the second sessionoccur during the same time, (ii) the second session is a conferencesession different from the first session, (iii) and the user isattending the second session as an observer.
 3. The method of claim 1,further comprising: displaying, by one or more computer processors, afirst graphical control element corresponding to the first session and asecond graphical control element corresponding the second session; anddisplaying, by one or more computer processors, at least a portion ofthe streaming text transcript corresponding to the second session withina respective proximity of the first graphical control elementcorresponding to the second session.
 4. The method of claim 1, whereindetecting the at least one trigger of the identified one or morepredefined triggers, within the second session of the plurality ofconference sessions that the user is attending, further comprises: inresponse to detecting the at least one trigger of the one or moreobtained triggers within the streaming text transcript corresponding tothe second session, applying, by one or more computer processors, avisual indication to the second graphical control element associatedwith the second conference session and a portion of the streaming texttranscript corresponding to the second conference session that includesthe detected at least one trigger.
 5. The method of claim 1, whereingenerating the streaming text transcript based on content spoken duringthe conference session and at least one element of interpreted contentthat occurs during the conference session further comprises:identifying, by one or more computer processors, a visual elementthrough real-time analysis of the presentation corresponding to theconference session that the user is attending; in response toidentifying the visual element, determining, by one or more computerprocessors, through one or more real-time analyses, a text descriptionrelated to the identified visual element corresponding to the conferencesession; and embedding, by one or more computer processors, thedetermined text description related to the identified visual elementwithin the generated streaming text transcript corresponding to theconference session.
 6. The method of claim 5, wherein embedding thedetermined text description related to the identified visual elementwithin the generated streaming text transcript corresponding to theconference session further comprises: including, by one or more computerprocessors, within the determined text description related to theidentified visual element, a hyperlink to an information resourceassociated with the identified visual element.
 7. The method of claim 1,further comprising: identifying, by one or more computer processors, oneor more visual elements within the presentation corresponding to theconference session that the user is attending through real-time analysisof the presentation; determining, by one or more computer processors,based on a cognitive analysis, whether a visual element of theidentified one or more visual elements corresponding to the conferencesession relates to one or more predefined triggers respectivelyassociated with the conference session; and in response to determiningthat the cognitive analysis of the visual element corresponding to theconference session relates to trigger of the one or more predefinedtriggers respectively associated with the conference session,determining, by one or more computer processors, that a trigger of theone or more predefined triggers respectively associated with theconference session is detected.
 8. The method of claim 1, furthercomprising: determining, by one or more computer processors, that theuser selects the interface corresponding to the second session; and inresponse to determining that the user selects the interfacecorresponding to the second session, increasing, by one or more computerprocessors, a size associated with the first graphical control elementthat displays a presentation corresponding to the second session to theuser; and modifying, by one or more computer processors, a connection ofthe user to the second session, wherein modifying the connection enablesthe user to attend the second session as an active participant.
 9. Themethod of claim 8, further comprising: modifying, by one or morecomputer processors, a connection of the user to the first session,wherein modifying the connection enables the user to attend the firstsession as an observer.
 10. The method of claim 5, wherein determining atext description related to the identified visual element correspondingto the conference session based on the one or more real-time analyses isselected from the group consisting of a cognitive analysis, imagerecognition, image processing, gesture recognition, and contextualanalysis.
 11. The method of claim 7, wherein an automated cognitiveanalysis utilizes information derived from one or more analyses selectedfrom the group consisting of image recognition, image processing,gesture recognition, and contextual analysis.
 12. The method of claim 4,further comprising: modifying, by one or more computer processors, thevisual indication applied to the second graphical control elementassociated with the second conference session is modified based on anumber of unique triggers that are detected.
 13. The method of claim 4,further comprising: modifying, by one or more computer processors, thevisual indication applied to the second graphical control elementassociated with the second conference session is modified based on thefrequency of occurrence of a detected trigger.
 14. The method of claim4, further comprising: modifying, by one or more computer processors,the visual indication applied to the second graphical control elementassociated with the second conference session is modified based on ananalysis of a detected trigger.